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Abstract 

This note introduces a new analytic approach to the solution of a very general class of 
finite-horizon optimal control problems formulated for discrete-time systems. This 
approach provides a parametric expression for the optimal control sequences, as well 
as the corresponding optimal state trajectories, by exploiting a new decomposition 
of the so-called extended symplectic pencil. Importantly, the results established in 
this paper hold under assumptions that are weaker than the ones considered in the 
literature so far. Indeed, this approach does not require neither the regularity of 
the symplectic pencil, nor the modulus controllability of the underlying system. In 
the development of the approach presented in this paper, several ancillary results of 
independent interest on generalised Riccati equations and on the eigenstructure of 
the extended symplectic pencil will also be presented. 
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1 Introduction 



This paper focuses on finite-horizon LQ problems with a generalised cost function and with affine 
constraints at the end-points. These problems are not just important per se. In fairly recent literature it 
has been shown that LQ problems are becoming increasingly useful as building blocks to solve complex 
optimisation problems, broken down into two or more LQ subproblems, each one with constraints at the 
end-points. A typical example is the ^-optimal tracking problem of previewed reference signals, which 
can be solved by splitting the problem into two coupled LQ problems, 0. Finite-horizon LQ problems 
with constraints at the end-points are also useful in the solution of other optimisation problems, including 
H2 receding-horizon problems and the minimisation of regulation transient in switching linear plants. 

The aim of this paper is to present a method to solve the most general class of finite-horizon linear- 
quadratic (LQ) optimal control problems in the discrete time with positive semi-definite cost index and 
affine constraints at the end-points. The approach taken in this paper is based on a procedure for the 
parameterisation of the set of trajectories generated by the so-called extended symplectic difference 
equation (ESDE). The idea of solving finite-horizon LQ problems by exploiting expressions of the 
trajectories generated by the Hamiltonian system in the continuous time or the ESDE in the discrete 
time originated in the papers B3, lfl4ll and for the continuous time, and in Q and 0-J for the discrete 
time. In both situations, the expressions parameterising the trajectories of the Hamiltonian system and 
the symplectic equation hinge on particular solutions of the associated continuous/discrete algebraic 
Riccati equations and on the solution of the corresponding closed-loop continuous/discrete Lyapunov 
equation. While controllability of the given system was required in the first papers El 0, because 
both the stabilising and antistabilising solutions of the ARE were involved, in more recent times it has 
been shown that generalisations of the same technique are possible under the much milder assumption 
of sign-controllability in the continuous case and modulus controllability in the discrete case, see 0. 
In a subsequent paper [fT8ll the same problem was considered under the more restrictive assumption of 
stabilisability. Some problems in the solution presented in [fT8l have been analysed and corrected in 0. 
The assumptions of sign/modulus-controllability (or stabilisability) were needed in the above-mentioned 
papers because the solution presented there was based on the existence of a solution of the closed-loop 
Lyapunov equation. In the discrete case, the other standing assumption was the regularity of the extended 
symplectic pencil. The goal of this paper is to propose a new approach aimed at overcoming these 
limitations. More precisely, in this paper a direct method is developed which generalises the technique 
in and flU in two directions. First, we do not require the symplectic pencil to be regular, nor to 
have a spectrum devoid of eigenvalues on the unit circle. As such, with the method proposed in this 
paper, regular and singular problems can be tackled in a unified manner. Second, unlike the other 
contributions on this topic, the method presented in this paper does not involve the solvability of the 
closed-loop Lyapunov equation. Therefore, even the modulus controllability assumption can be dropped. 
The technique presented in this paper only requires a solution of the so-called generalised discrete-time 
algebraic Riccati equation, which may exist even when the symplectic pencil is not regular (while in this 
case the standard discrete algebraic Riccati equation cannot be solved). Such solution is used to derive 
a decomposition of the extended symplectic pencil that yields a natural parameterisation of the solutions 
of the symplectic difference equation. A large number of LQ problems dealt with in the literature by 
resorting to different - often iterative - techniques can be tackled in a unified framework and in finite, 
nonrecursive terms, by means of the method developed in this paper. For a better description of the 
features and the generality of our framework, we illustrate all our results in a running example in which 
the underlying system is not modulus controllable, and the extended symplectic pencil is not regular (so 
that the methods in previous literature cannot be used). 
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2 Statement of the problem 



Consider the linear time-invariant discrete-time system governed by the difference equation 

x(t + l) =Ax(t) +Bu(t), 

where, for all t G N, x(t)eW is the state, u(t) GlR m is the control input, A G W 1 x " and Be 
T G N \ {0} be the length of the time horizon. Let V , V T G x " and v G M ? ; consider 

v x(o)+v r x(r)=v, 



(i) 

r xm . Let 



(2) 



which represents a two-point boundary- value affine constraint on the states at the end-points. With no 
loss of generality, we can consider V = [Vb Vj] to be of full row rank. In the case where ^ = 0, the matri- 
ces Vb, Vt, V and the vector v are considered to be void: in this case © does not constrain any component 
of the initial and terminal states. 



Q S 

.S T R 

(note that we do not assume the non- singularity of R). We denote by E the Popov triple 



Let n = 
ReR mxm 
(A,B,U), Finally, let// 



n T > be a square (n + m) -dimensional matrix with QEM. nxn , SEW lxm and 



Hi H 2 

L# 2 T 



H T >0 with ^1,^2,^3 el 



and hQ.hxE. 



Problem 2.1 Find u(t), t G {0, . . .,T — 1} and x{t), t G {0, . . . , T}, minimising 



r-i 



7(jc,h) = ^ [x T (0 u T (t)]ll 



t=0 



x(t) 
u(t) 



+ [x T {Q)-hl x T (T) — hj J H 



x{0)-h 
x(T)-h T 



(3) 



under the constraints (ClEI). 



The formulation of Problem l2.1l is very general, since the cost index in © involves the most general type 
of positive semidefinite quadratic penalisation on the extreme states, and Q represents the most general 
affine constraint on these states. As particular cases of Problem l2.1l we have: 

• the standard case where x(0) is assigned and x(T) is weighted in ©; this case can be recovered 
from Problem O by setting V = I n , V T = 0, h T =0, Hi =H 2 = 0; 

• the fixed end-point case, where the states at the end-points are sharply assigned; this case can be 
recovered from Problem [2TT1 by setting V —h n and H = 0; 

• the point-to-point case, where the extreme values of an output y(t) =Cx(t) are constrained to be 



equal to two assigned vectors yo and yr, by taking V = diag(C, C), and v 



yo 
yr 



Further non-standard LQ problems that can be useful in practice are particular cases of Problem |2T1 con- 
sider for example an LQ problem in which the states at the end-points x(0) and x(T) are not assigned, but 
they are constrained to be equal, i.e., x(0) = x(T). This case can be obtained by Problem [2TT1 by setting 
V = I n , V T = -I n and v = 0. 
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Lemma 2.1 [4, Lemma 3] If u{t) and x(t) are optimal for Problem I2.il then X (t) e W\ t£{0,...,T} 
and r)GM s exist such that x(t), X (t), u(t) and r\ satisfy the set of equations 



x(t + l) =Ax(t)+Bu(t) te{0,...,T-l}, 

x(0) 
x(T) 

X(t) = Qx(t) +A T X (? + !)+ Su(t) te{0,...,T-l}, 



V 



-X(0) 
X(T) 



H 



x(0)-h 
x(T)-h T 



+ V T r, : 



= S T x(t)+B* X(t + l)+Ru(t) te{0,...,T-l}. 



(4) 
(5) 
(6) 
(7) 

(8) 



Conversely, if equations (EHSj) admit solutions x(t), u(t), X(t), r\, then x(t), u(t) minimise J(x, u) subject 
to the constraints (HE]). 

The variables X (t) in (HH8]) represent the Lagrange multipliers associated with the constraint (QQ), lfT2l[T0ll . 
while the variable r\ £ W is the Lagrange multiplier vector associated with ©. 



3 The generalised Riccati equation and the extended symplectic 
system 

Since in the present setting we are not assuming that R is positive definite, ([8]) cannot be solved in u(t) to 
obtain a set of In equations in x(t) and X(t). A convenient form in which (HJ), © and © can be written, 
that does not require inversion of R, is the descriptor form 

Mp(t + l)=Np(t) ?G {0,...,r-l}, (9) 



where 




'In 





' 




' A 





B ' 




x(t) 




M = 





—A T 







Q 


-In 


S 


, P(t) = 


X(t) 









-B T 







s T 





R 




u(t) 



The matrix pencil Af — zM is known as the extended symplectic pencil, lfT2l[T0ll . herein denoted concisely 
by ESP(E). In this paper we do not make the assumption of regularity of this pencil. 

We now show how a solution of a generalised discrete algebraic Riccati equation can be used to obtain 
a decomposition of ESP(E) that can be used to solve Problem 12.11 In particular, we will exploit the 
solutions of the following constrained matrix equation 

X = A T XA-(A T XB + S)(R + B T XB) t (B T XA + S T )+Q, (10) 
ker(R + B T XB) C ker(A T X5 + 5), (11) 

where the matrix inverse that appears in the standard discrete algebraic Riccati equation (DARE) has been 
replaced by the Moore-Penrose pseudo-inverse. Eq. (flOl) is known in the literature as the generalised 
discrete-time algebraic Riccati equation GDARE(E), [fT5l l9l. GDARE(E) with the additional constraint 
given by (fTTT) is sometimes referred to as constrained generalised discrete-time algebraic Riccati equation 
CGDARE(E). Clearly (flOl) constitutes a generalisation of the classic DARE(E), in the sense that any 
solution of DARE(E) is also a solution of GDARE(E) - and therefore also of CGDARE(E) - but the 
vice-versa is not true in general. 
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We now introduce some notation that will be used throughout the paper. First, to any matrix X = X T e 
l nxn we associate the following matrices: 



S x 4 A T XB+S : R X =R+B T XB, G X =I„ 



R x Rx, 



K x 



{R + B T XB) r (B T XA + S T )=R x Sj c , A x =A-BK X . 



(12) 
(13) 



The term R x Rx is the orthogonal projector that projects onto imR x = imR x so that G x is the orthogonal 
projector that projects onto kerR x . Hence, kerR x = imGx- 

Since as aforementioned the Popov matrix n is assumed to be symmetric and positive semidefinite, 

we can consider a standard factorisation of the form n 

R = D T D. 



LD T J 



[c d], where Q = C T C, S = C T D and 



Example 3.1 Consider the following Popov triple, which will be used as a running example throughout 
the paper: 



"11" 


, B = 


"20" 


, Q = 


"00" 




" 


" 


, R = 


" 


" 


1 


1 1 


1 


, s = 















The Popov matrix IT in this case can be factored with C = [0 1 ] and D = [0 0]. The extended symplectic 
pencil in this case is not regular. As such, DARE(L) in this case does not admit solutions. On the other 
hand, in this case CGDARE(L) admits the solution X = diag{0, 1}, that can be computed by resorting to 



the algorithm proposed in £7]/. In this case, R x = R + B T X B 



l l 
l l 



, and the corresponding closed-loop 



matrix is Ax = diag{ 1 , 0}. Observe that the spectrum of Ax is not unmixed. 



□ 



Let Cx — C — DR X S X , and let Mx denote the reachable subspace associated with the pair (A,BG x ), 
in symbols M x = im[BG x A X BG X A 2 X BG X ... A n x l BG x }. 

The following results were proved in [6, Lemma 4.1, Lemma 4.2, Theorem 4.2]. 

Lemma 3.1 Let X =X T be a solution of CGDARE(H). Then, 

(i) M x C kerC x ; 

(ii) kertf* = ker(Xfl) nkeri?; 

(iii) £%x is the reachability subspace on the output-nulling subspace kerX. 

We also have the following results, see J6l Theorems 4.3-4.4]. 

Lemma 3.2 Let X and Y be two solutions of CGDARE(L). Let Ax and Ay be the corresponding closed- 
loop matrices. Then, 

(i) kerRx = kerRy; 

(ii) ^ x = ^y; 
(m)A x y x =A Y y Y . 

The following result adapts (H Lemma 2.5] to the case when the matrix pencil N — zM may be singular. 
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Lemma 3.3 Let X = X T be a solution of CGDARE(L ). Then, Ux , Vx £ M. 2n+m exist such that 

B 



U x (N-zM)V x 
Proof: By direct computation we find 



Ax-Zh O 
O I n 

O -zB T R x 



■zAl O 





'Ax-zh 


B ' 




' In 





' 




' In 





0~ 


Ux(N-zM)V x = 


S 2 1 In-ZAI 


£23 


with Ux — 


A X X 


In 






X 


-In 







77 T - 7 R T 






B T X 





Im 




-K x 





Im 



(14) 



The term S21 is given by 

S 2 i = A T X XA -A T X XBK X + Q-X- SK X - K^S T + K£RK x -z (A t X -A T x X+KlB T X). 

The term multiplying z is zero since Ax = A — BKx- Moreover, since GDARE(E) can be written as 
X = A T XA -SxKx + Q we find S 2 i = K£(R X K X - S£) = S x R x RxR x Sx - $xR x s x = °- Finally, 
S 23 =A T XB - zXB - K%B T XB + S + zXB - K$R = S x G x . In view of ([H]), we have S x G x = 0, so 
that CH]) holds. ■ 
If X is a solution of CGDARE(E), from the triangular structure in (PT41) we have 



det(iV-zM) = det(A z -z/ n )-det(/ n -zAx)-detR z . 



(15) 



When Rx is non-singular (i.e. X is a solution of DARE(E)), the dynamics represented by this matrix 
pencil are decomposed into a part governed by the generalised eigenstructure of Ax — zl n , a part governed 
by the finite generalised eigenstructure of /„ — zA x , and a part which corresponds to the dynamics of the 
eigenvalues at infinity. When X is a solution of DARE(E), the generalised eigenvalue^] of N — Mz are 
given by the eigenvalues of Ax, the reciprocal of the non-zero eigenvalues of Ax, and a generalised 
eigenvalues at infinity whose algebraic multiplicity is equal to m plus the algebraic multiplicity of the 
eigenvalue of Ax at the origin, and we have 



o(N - zM) = o(A x - zl n ) U o 



In ^Ajf 

-zB T 



o 

Rx 



(16) 



When the matrix Rx is singular, (PT5T) still holds but provides no information as in this case detRx 
while (fT6l) is no longer true. We show this fact with a simple example. 



0, 



Example 3.2 Consider Example I3.il Matrix X = diag{0, 1} is a solution of CGDARE(L), and the 
corresponding closed-loop matrix is Ax = diag{l,0}. From Lemma \33\ we find 



U X (N- 





' l-z 











2 


" 







— z 








1 


1 


zM) V x = 








l-z 




















1 
















-2z 


— z 


1 


1 













—z 


1 


1 



Recall that a generalised eigenvalue of a matrix pencil — zM is a value of z G C for which the rank of the matrix pencil 
jV — zM is lower than its normal rank. 
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whose normal rank ( which coincides with that ofN — zM) is easily seen to be equal to 5. The eigenvalues 
of Ax are and 1. However, it is not true that z = 1 is a generalised eigenvalue ofN — zM. In fact, a 
direct check shows that the rank ofN — M is equal to 

Consider a change of coordinates in the input space W induced by the m x m orthogonal matrix 
T = [T\ T 2 ] where imTi = imi?x and im^ = imG^ = keri?^- From Lemma [3721 T is independent of 
the solution X of CGDARE(E). Thus T T Rx T = diag{i?x,o 5 O}, where R x ,o is invertible. Its dimension 
is denoted by mi. Consider the block matrix f = diag(/„,/„,r). Defining the matrices B\ = BT\ and 
B2 = B T2 we get 



f T (Ux(N-zM)V x )f 



Ax-Zln 

O 

o 
o 



o 

n Z.A^ 

-zBj 
-zBl 



B\ 
O 

Rx,o 
o 



B 2 
O 
O 
O 



In view of kerRx = imG^, we get im^2 = im(5 Gx)- Matrix B\ has m\ columns. Let mi — m — m\ be the 
number of columns of B 2 - Let us take U = [U\ U2} such that imU\ is the reachable subspace associated 
with the pair {Ax^Bi), which coincides with the subspace Six- We have 



U~ l A x U 



Ax, 11 
O 



A x ,n 
Ax, 22 



U~ l B 2 



B21 
O 



U l B { 



Bn 

B\2 



(17) 



Lett/ = diag{t/,[/,/ OTl ,/ OT2 }. Let r denote the size oi&x- Defining the two unimodular matrices 



^1 



Ir 























Ir 


























lyni 





lyi—r 























In—r 




















Im\ 






and ^2 



h 


























In—r 














Ir 























In—r 




















I in I 





I\ni 















we get 



P(z) = n l U- l f T (U x (N-zM)Vx)fUQ 2 = 



' A X ,U -Zl r 


B21 





A X ,12 





Bn ' 








I r — zA^ n 

















-zBl 




















Ax, 22 - Zln-r 





B\2 








~ zA X,12 





I n -r — zAg 22 











-zBJi 





-ZBl2 


Rx,o . 



(18) 



Since the pair (Ax, 11,-621) is reachable by construction, all the r rows of the submatrix [A^.n — zl r B21 } 
are linearly independent for every z G C U {°°}. This also means that of the r + m2 columns of [Ax.n — 
zl r B21 ] 5 only r are linearly independent, and this gives rise to the presence of a null-space of P(z) whose 

2 We warn that the routine eig.m of the software MATLAB® (version 7.11.0.584(R2010b)) in this case fails to provide 
the right answer. It indeed returns 1 as a generalised eigenvalue of the pencil N — zM. 



6 



dimension «?2 is independent of z G C U {°°}. We obtain^ 



rankP(z) 





I r zA xn 













-zBl x 











r + rank 





Ax,22—Zl n -r 





B\2 




~ Z ^X\2 





In-r — zA x 22 












-zBj 2 


Rx,o 



Now, consider the rank of 



7 A T 

ZA X,U 



zB 



21 



Again, since the pair (Ax \ 1,-621) is reachable, this rank is constant 



and equal to r for every zGCU {°°}- Thus, 



rank,P(z) = 2r +rankPi(z), where Pi(z) 



Ax,22-zl n -r 

o 
o 



In r 



o 

— zA T 
zB T 



x.22 



12 



B12 
O 

Rx,o 



Since detPi(z) = det(A x . 22 — zl n - r ) ■ det(/ n _ r — zA x 22 ) ■ detR x .o, a value z G C can be found for which 
detPi(z) 7^ 0. Hence, the normal rank of P\(z) is equal to 2 (n — r) +m\, and therefore the normal rank 
of P(z) is 2 r + 2 (n — r) + mi =2n + m\. The generalised eigenvalues of the pencil P(z) are the values 
z G C U {00} for which the rank of Pi (z) is smaller than its normal rank 2 (n — r) +m\ . These values are the 
eigenvalues of A^,22 plus their reciprocals, included possibly the eigenvalue at infinity, whose multiplicity 
— be it algebraic or geometric — is, in general, not given by the sum of m\ plus the multiplicity of the 
eigenvalue in zero of A^,22- In fact, the multiplicity of the eigenvalue at infinity is the multiplicity of the 
zero eigenvalue of 





Ifi—r 








p - 





A 1 

X,22 










5 12 






If Ax ,22 is non-singular, the last mi columns give rise to an eigenvalue at infinity whose multiplicity (al- 
gebraic and geometric) is exactly equal to mi, since in this case the dimension of the null-space of P x is 
equal to m\. However, if Ax, 22 is singular, the algebraic (geometric) multiplicity of the zero eigenvalue 
of Poo is equal the sum of m\ plus the algebraic (geometric) multiplicity of the eigenvalue in zero of A x 22 
that is non-observable for the pair (A x 22^12) an( ^ mes e are indeed the multiplicities of the eigenvalue at 
infinity of the pencil. 

From these considerations, it turns out that, unlike the regular case, not all the eigenvalues of Ax appear 
as generalised eigenvalues of ESP(E). In particular, the eigenvalues of Ax restricted to Mx do not appear 
as generalised eigenvalues, whereas the eigenvalues of the map induced by Ax in the quotient space 
M"/Mx - along with the reciprocals of those that are different from zero - are generalised eigenvalues of 
ESP(E). 



Example 3.3 Consider Example \3J\ Using the solution X = diag{0, 1} of CDARE(L) we get kerR x 



and imRx 



By taking T 



m i = m 2 = 1- We partition BT as BT 



1 -1 

1 1 

"2 -2 

2 



we obtained T T Rx T = diag{4, 0}. Hence, in this case 



, so that B\ 



and B2 



-2 




The normal rank of 



3 Let E = 



-11 "12 
O H 22 



Observe that if either En is full row-rank or E22 is full column-rank, then rank E = rank E 1 1 + rank E22 ■ 
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ESP(L) is equal to2n J r m\ = 5. The generalised eigenvalues ofN — zM are given by the uncontrollable 
eigenvalues of the pair (Ax, #2) = ( qq , ~0 ) ^ us tne ^ r reciprocals. Therefore, ESP(L) has a 
generalised eigenvalue at the origin. Since Ax 22 = and B\ 2 = 2, it also has an eigenvalue at infinity 



with multiplicities equal to the multiplicities of the zero eigenvalue of 
form given by flM . we get 




2 



. By writing this pencil in the 



f T {Ux{N-zM)V x )f 



' l-z 


-2 











2 ' 








l-z 

















2z 




















— z 





2 














1 











-2z 





-2z 


4 



from which we see that zero is indeed the only finite generalised eigenvalue ofESPfL). 



□ 



4 Solution of the LQ problem 

In the basis constructed in the previous section, © can be written for t £ {0, 



T-l} as 



XI + 


= A X ,uX ]L (t)+B 2 ).Ui(t)+Ax,l2X2{t)+B ]Ll U2{t), 


(19) 


h(t) 


= AlnW + 1), 


(20) 





= -B^X^t+l), 


(21) 


x 2 {t + l) 


= Ax,22X2(t)+Bi2U 2 (t), 


(22) 


hit) 




(23) 


1*2(0 


= /?- 1 fi 1 T 2 ^(?+l)+ J R- 1 B 1 T 1 X 1 (? + l). 


(24) 



Since by construction the pair (Ax.i 1,-821) is reachable, ker ' = {0}, which means 

L S 21 J 

X\ (t) = for all t e {0, . . . , T - 1}. This implies that (|23H24I) can be simplified as 

A 2 (0 = Al 22 h(t + 1), 
u 2 (t) = R x \B T n X 2 {t + \). 



yield 

(25) 
(26) 



It is clear at this point that we can parameterise all the trajectories generated by the difference equations 
(1221) . (1251) and © in terms of x 2 (0) and X 2 (T). Indeed, d23) leads to 



Ut) = (Al 22 ) T - t h(T) WG{0,...,r}. 

This expression can be plugged into (1261) and leads 

U2(t)=R x l Bj 2 (Al 2 2) T - t - l h(T). 
Plugging ([27]) and (EH) into d22]) gives 

X2(0=A^ 22 X2(0) + £A^- 1 fii 2 7? x J ) fi 1 T 2 (A^ 22 ) r ^- 1 A2(r) 

7=0 



(27) 



(28) 



(29) 
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It is worth observing that 



r-i 



x 2 (r)=A| )22 x 2 (0)-PA 2 (T), where P 4 £aJ/ 2 1 B 12 ^ ) 1 B 1 T 2 (A^ 22 ) r ^'- 1 . 



(30) 



It is easy to see that matrix P can be re-written as P = Lj=c i A?r 22 ^l2^y V*i2(Ax 22) J - Therefore, P 
satisfies the discrete Lyapunov equation 

^ = Ax,22^A^22 -^l,22 5 12^,0 5 12 (Af,22) + fi 12#jf ) B 12- 

If Ax, 22 has unmixed spectrum, this equation can be used to determine P instead of computing the sum 
in (|30l) . At this point we can solve (fT9l) . which can be written as 

xiO+1) = A x ,ux 1 (t)+B 2 iui(t) + Z(t), (31) 

where £(t) =Ax,i 2 x 2 (f) +B n u 2 {t). Using ([29]) and (ED) we find 

£(0=^ 12 A^ 22 x 2 (0)+^ 



Let 7?! = [B 2 i |A Z) nfi 21 \A 2 xn B 2 i \ ■ ■ ■ | A^}fl 2 i] and 7? 2 = [/ | A x ,u \ A 2 xn \ ■■■ |A^|]. Then, we 

'ui(r-i) 



1(0) 



and {/1 



can write xi(r) =A^ ^^1(0) +2? 2 S+/?i C/i where S 

T is greater than the controllability index of the pair (A x ,n,5 2 i). All the so 
parameterised by 

U X = R\ (xi (T) - A£ n xi (0) - tf 2 E) + (/ - R\ R X ) vj . 

where vi is arbitrary. 



We assume that 

m(0) 

utions of this equation are 



(32) 



4.1 Boundary conditions 

Consider the change of coordinates given by the matrix U = [U\ t/ 2 ], where imU\ is the reachable 



subspace of the pair (Ax ,BGx)- Let 



xi (f) 

.*2(0 



£/ 1 be the coordinates of the state in the basis induced 



by U, partitioned conformably with U. The state, co-state and transversality equations can be written 
again as in ©, © and ©, where A, B, Q, S, V, H, h and h T are replaced by U~ l AU, U 1 B, U T QU, 

U T S, V 



u o 
o u 



u o 
o u 



H 



u o 
o u 



, U l ho and U hr, respectively. We can now write © and © with 

respect to this basis. We can eliminate the multiplier r\ from © by premultiplying both sides of this 
equation by a basis Ky of kerV: 



*(0) 
x(T) 



I O 

O -I 



A(0) 
X(T) 



KyH 



h 
hj 



(33) 



In this way, © and (1331) can be written together as a set of 2n linear equations in jc(0), x(T), A(0) 
and A(r). However, in (1301) the component x 2 (T) is expressed as a linear function of x 2 (0) and A 2 (T), 
and A 2 (0) can be expressed as a linear function in X 2 (T) by (|27l) . Finally we know that X\(t) must be 
identically zero, so that Ai(0) = X\(T) = 0. Therefore, in this basis © and © can be expressed as a 
single linear equation of the form 

Fx = g, where x=[xj(0) xJ(T) x 2 T (0) A 2 T (r)] T . (34) 

We have just proved the following result. 
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Theorem 4.1 Problem I2.il admits solutions if and only if Utfb does. For any solution x 



[xf(0) x\{T) xj(0) X 2 T {T)] T we get an optimal initial state x(0) 



(0) 

1*2(0) 



and a class of optimal 



controls parameterised by rt2ffl) and 021) . The solutions obtained in this way are all the solutions of 
Problem \2.1\ 

Example 4.1 Consider a finite-horizon LQ problem in the time interval {0, . . . , T}, involving the matri- 
ces given in Example I3.il The initial and final states are constrained to be equal, i.e., x(0) = x(T). Let 

and hj = 0. As aforementioned, X = diag{0, 1} is a solution of CGDARE(L), leading 



H = I 2n , ho 



to Ax = diag{ 1,0}. By taking T 



BT 



2 -2 
2 



, so that B\ 



l -l 
l l 

and B 2 



, we obtained T T Rx T = diag{4, 0}, so thatRo x = 4. Recall that 
Therefore, the reachable subspace of the pair (Ax,B 2 ) 



is lm 
B n = 



, which means this system is already in the desired basis. Thus, Ax n = 1, Ax 12 = Ax 22 = 0, 



B\2 = 2 and B 2 \ = —2. In this case, KT2\ . ([25]) and rf26l) become 
Thisim P x dsiha} = B ^2(t), hif) = 0- X 2 (t + 1), u 2 (t) = 



R Xfi B i2^2(t +1). 



which give 



t e {o,...,r-i} 

HT) t = T, 



u 2 (t) 







f e{0,...,r-2} 



R x l B{ 2 X 2 (T) t = T-l, 



x 2 (0) 
x 2 (t)={ 0^ 



t = 
t = T. 



T-l} 



In this basis, © gives rise to xi(0) = x\{T) and x 2 (0) = x 2 {T) = X 2 (T), which are linear in x\{T) and 
X 2 {T), while fi33\) can be written asxi(0) + xi(T) = hi and x 2 (0) +x 2 (T) + ^(0) —X 2 (T) = h 2 . Since 
X 2 (0) = andx 2 (T) = X 2 (T), the latter can be written as x 2 (0) — h 2 . Therefore, the boundary conditions 
can be written in the form fi34\) : 



1 





-1 


" 




" *i (0) " 




' 





1 





-1 




x 2 (0) 







1 





1 







X1 (T) 




hi 





1 










X 2 {T) 




h 2 



This linear equation admits only the solution x\(0) = x\(T) = h\/2 and x 2 (f)) = X 2 (T) = h 2 . Now we 
can compute the optimal control law. First, u 2 (t) is zero for all t e {0, . . . , T — 2} and u 2 (T — 1) = 
R x l Q Bl 2 X 2 (T) = h 2 /2. In order to compute u\, we write rtiPl) as 



xiO + l) = l-*i(f)-2ui(f) + £(0- 



(35) 



The term £, (t) in this case is equal to zero for all t £ {0, . . . ,T — 2} and t, (T — 1) = BnR QX Bl 2 X 2 (T) 
X 2 (T) = h 2 . We can write ( 1321) explicitly as 

r ' " r u\{T- 



x l (T)=x l (0) + 



I A 



l XAl 



h 2 





+ [ -2 -2 ... -2 ] 



T 



1) 

m(T-2) 

ui{0) 



which gives 
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Ki(r-i) 
m(T-2) 

ui{0) 





" 1 ~ 




1 








. i . 



+ 



l-r o 

1 2-T 








where v is arbitrary and represents the degree of freedom in the control u\. 



□ 



Remark 4.1 So far, we have not considered the problem of existence of solutions for Problem \2.1\ In 
general, the existence of a state trajectory x{t) satisfying the constraints (EKP/or some u{t) is not ensured, 
since we have not assumed reachability on (0). A necessary and sufficient condition for the existence of 
optimal solutions is that there exist state and input trajectories satisfying (H|2p (feasible solutions). In 
fact, since the optimal control problem formulated in Section \2\ involves a finite number of variables - 
precisely, L = m- T for the control plus nfor the initial state - Problem \2~T\ can be restated as a quadratic 
static optimization problem in these L + n variables with linear constraints. Thus, a solution to Problem 
\2.1\ exists if and only if a feasible solution - i.e., a state and input functions satisfying both (0) and © - 
exists. 



Remark 4.2 The approach presented in this paper can successfully tackle even more general LQ prob- 
lems, where the performance index is not necessarily positive semidefinite. E.g., consider 

T-l 

J( x ,u) = £ [x T (t) u T {t)]n 

t=0 

Although all the variational analysis remains unaffected, the presence of the term 2C,x{T) deserves 
some considerations. Indeed, this linear term may cause the divergence to —°° of the cost index in 
correspondence to a sequence of admissible controls so that, even in the presence of feasible solutions, the 
optimal control may fail to exist^ In this case, the linear equation representing the boundary conditions 
is infeasible. Two simple a priori sufficient conditions for the existence of the optimal control and hence 
for the solvability of the two-point boundary-value problem are the following: 

1. kerH C ker £ T . Under this condition, the cost on the final state (and hence the overall cost index) 
is bounded from below. Indeed, such a cost may be rewritten as a constant plus a positive semi- 
definite quadratic form (x(T) —x) T H(x(T) —x) in the difference between x[T) and a suitable 
"target state" x. In this case the solution of the problem indeed exists. 

2. R > 0. In this case the current cost increases quadratically with the norm of the control input with 
the largest norm and, in the best situation, decreases linearly with the same norm. Thus the search 
for the optimal control input can be restricted to a compact set in M. mxT and hence the optimal 
solution does exist. 

4 Consider for example the case where A, B and Q are the 2x2 identity matrices, while R, S and H are the zero matrices and 
£ = [ 1 1 ] T . For this system, the LQ problem in one step (i.e., T = 1) has no solution; in fact, the control m(0) = — x(0) —m^ 
yields a value of the cost which goes to — °o as the parameter m goes to +°°. 



x(t) 
u(t) 



+ x T (T)Hx(T) + 2C T x(T) 
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5 Conclusions 



In this note, we studied the discrete-time finite-horizon LQ problem with the most general type of 
positive semidefinite cost function and with affine constraints at the end-points. We derived an analytic 
approach, based on a special decomposition of the extended symplectic pencil, that generalises several 
contributions that have appeared in the literature in the last few years on this problem. Indeed, this 
approach does not require regularity of the extended symplectic pencil, nor the modulus controllability 
of the underlying system. Due to its generality, the proposed technique can be used to efficiently tackle 
complex optimisation problems of wide interest, including the ^-optimal rejection/tracking of previewed 
signals, receding-horizon optimal control problems, and the minimisation of regulation transients for 
plants subject to large parameter jumps. 
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